A Computer Music Implementation Course Using Active Essays

Computer music tends to be the domain of musicians, electrical engineers, mathematicians, and physicists. Today, only CS1-level knowledge is necessary to do a serious review and exploration of computer music, as the algorithms have become more well understood (and thus easier to explain), and Moore’s Law makes even inefficient algorithms successful. A serious impediment, however, has been the lack of a text that is written from a programming perspective (instead of mathematical or musical). By using active essays in Squeak, a set of notes have been created where students can read the code, execute it, hear the result, see the waveform, and take the Fourier transform. 1 The Domain of Computer Music Computer music is an unusual area of study in that the word “computer” is in the name, yet few Computer Science departments offer classes in it. Instead, computer music is often taught as part of digital signal processing courses in Electrical Engineering; perhaps as a special topic in music course; and perhaps as part of a physics course on the Physics of Music. There are several reasons for considering incorporating computer music into Computer Science curricula: • Music on computers is a growing common concern (rf. Napster), and appears across the Web in various guises. With such a pervasive medium, computer professionals will doubtlessly need to work with it. Knowledge of the variables and algorithms of computer music would better prepare our students for their careers. • Recent reports exploring why students drop-out (or completely avoid) computer courses suggest that part of the problem is simply the tedious, overly-technical assignments that we require which involve very little creativity [1]. Computer music offers the opportunity to teach about computer science using assignments that might engage a different kind of student and that do allow for a good bit of creativity. • While other departments may teach the underlying physics and mathematics of music and sound, or may teach how to use computer music applications, the implementation of the applications and the underlying algorithms for generating music is the domain of Computer Science. Existing computer music texts generally do not emphasize how to implement the underlying algorithms, which poses a problem for the Computer Science teacher who wishes to teach this content. There are many excellent books on computer music. Curtis Roads’ book [7] is a comprehensive tutorial over all of computer music, but with little on implementation. The new book on the popular music programming language Csound by Richard Boulanger [2] provides many examples of Csound programs (instrument definitions), but with little explanation of how the oscillators, resonators, and filters are implemented. Our favorite book for describing the algorithms [3] still does not provide examples of code. The most common computer music algorithms are not actually difficult to implement. The algorithms for oscillators, filters, and envelope generators are rather simple — they require no more than single (unnested loops) and fewer than a dozen lines of code. In this sense, they are well within the scope of a CS1/CS2 course. Of course, the simplest and easiest to understand implementations are not the most efficient. Moore’s Law enables even inefficient algorithms to generate reasonable sound samples in less than 30 seconds. Students can build interesting sound samples and explore new ones quickly enough to remain motivated. This paper describes a seven week course taught at Georgia Institute of Technology (Georgia Tech) by the authors, as a portion of a semester long sequence on Computer Music Implementation. While the overall course dealt with a wide range of issues in computer music (e.g., composition, improvisation, styles of music), the first half mini-course was focused just on the creation of tones in specific timbres. Very little prior knowledge of music was needed and only CS1-level programming knowledge was required. Thus, this short course is should be easily generalized to other schools in other kinds of curricula. A set of innovative multimedia notes were created to allow students to read, see, hear, and manipulate the example waveforms from within a standard browser. 2 Structure of Course The course was taught in the Fall 2001 semester as an elective undergraduate course. 20 students enrolled. Lecture and discussions took place in a room equipped with a computer and projector, but without adequate speakers. Instead, we hauled in a laptop computer and speakers for the course. The textbook for the course was Boulanger’s CSound book [2]. While the CSound book didn’t cover the algorithms we wanted to teach in sufficient detail, it did provide the next level up. CSound is a music language made up of unit generators that encapsulate many of the basic algorithms of computer music. CSound became the testing environment for the students—once they got something working in CSound, they could use that as a reference as they debugged their own code. The course was taught in Squeak [6], a cross-platform programming language based in Smalltalk which has excellent support for multimedia. A pre-requisite for the course was a required course using Squeak [5], but it would probably be possible to teach the course with students learning Squeak at the same time [4]. Squeak has built-in support for generating specific notes ((FMSound pitch: 100 dur: 0.5 loudness: 0.8) play. or (FMSound pitch: ’c3’ dur: 0.5 loudness: 0.8) play.), playing with specific timbres, or playing larger scale elements like scales ((FMSound lowMajorScaleOn: FMSound brass2) play), so its easy to do the musical equivalent of “Hello, World!” More important, though, is the support to build sounds from the digital ground up. Squeak allows the user to fill a sound buffer with raw integers and then send this sound buffer to a digital-to-analog converter for generating instruments. The course was structured to follow the historical progression from the earliest synthesis algorithms to the latest. Figure 1: Introductory Lecture to Squeak’s Sound Tools The flow of lectures was: 1. The first lecture introduced the tools that Squeak provides for studying sounds and music. Squeak supports recording sound, displaying waveforms, computing and displaying Fourier transforms, as well as displaying real-time Fast Fourier Transforms (FFTs) and Sonograms (Figure 1). 2. An introduction to acoustics and psychoacoustics explored the physical characteristics of sound (frequency, amplitude) and the relationship to the human perception of this sound. We did several in-class experiments, such as listening to tones very near one another in frequency (e.g., 100 Hz and 101 Hz, then 2000 Hz and 2001 Hz) to see when it was possible to notice a difference—and how that just noticeable difference became larger with higher frequencies. 3. We then did our first synthesis algorithm, additive synthesis where sine waves are simply added together. While additive synthesis generates unusual sounds, it is computationally intense and does not generate the richness of sounds like those found in natural instruments. 4. We then studied the sounds of natural instruments, using a tool called AudioExplorer (by Ph.D. student Jochen “Je77” Rick) which allows us to capture FFT in discrete slices. In this way, we were able to see how the spectral content of real instruments changed over time. 5. Next, we implemented an oscillator which could play any waveform at any desired frequency. This is a critical component (unit generator) used in most computer music synthesis systems. 6. We looked briefly at subtractive synthesis, which requires fewer computational resources than additive synthesis, but does require filters. We created a few digital filters using very simple algorithms. 7. We then introduced Frequency Modulation (FM) synthesis using our oscillator. FM Synthesis was the technique used in Yamaha’s popular synthesizers of the 70’s. Squeak also includes an implementation of FM synthesis, so two different implementations are compared. 8. Finally, we introduce synthesis based on sampling, which is the technique that most modern synthesizers use today.